ON THE EXISTENCE OF OPTIMAL SUBSPACE CLUSTERING 

MODELS 



AKRAM ALDROUBI AND ROMAIN TESSRA 



Abstract. Given a set of vectors F = {/i,...,/m} in a Hilbert space H, 
and given a family C of closed subspaces of H, the subspace clustering problem 
consists in finding a union of subspaces in C that best approximates (models) 
the data F. This problem has applications and connections to many areas 
of mathematics, computer science and engineering such as the Generalized 
Principle Component Analysis (GPCA), learning theory, compressed sensing, 
and sampling with finite rate of innovation, fn this paper, we characterize 
families of subspaces C for which such a best approximation exists. In finite 
dimensions the characterization is in terms of the convex hull of an augmented 
set C + . In infinite dimensions however, the characterization is in terms of a 
new but related notion of contact hull. As an application, the existence of 
best approximations from 7r(G)-invariant families C of unitary representations 
of abelian groups is derived. 

1. Introduction 

1.1. Motivation. Let H be a Hilbert space, F = {/i, . . . , f m } as set of vectors in 
H,Ca family of closed subspaces of %, V the set of all sequences of elements in C of 
length I ( i.e., V = V(Z) = {{Vi, . . . , Vi} : Vi G C, 1 < % < l}), and F a finite subset 
of H. The following problem has several applications in mathematics, engineering, 
and computer science: 

Problem 1 (Non-Linear Least Squares Subspace Approximation). 

(1) Given a finite set FcM, find the infimum of the expression 

e(F,V) := V mm d 2 (/,^), 

L — ' 1<7<( 
/£P - J - 

over V = {Vi, . . . , V;} g V, where d[x, y) := \\x — y\\n- 

(2) Find a sequence of subspaces V° = {V°, . . . , Vf} € V (if it exists) such that 

e (F, V°) = inf {e(F, V) : V g V}. (1.1) 



When H = R , and C is the family of all subspaces of dimension less than or equal 
to r, and 1 = 1, Problem[T]becomes the classical problem of searching for a subspace 
V of dimension dim(V) < r that best fit a (finite) set of data F C R d . For this 
case, the best approximation V exists and it can be obtained using the Singular 
Value Decomposition (SVD). 

When I > 1, we get a non-linear version of the single subspace approximation prob- 
lem mentioned above. This non-linear version also has many applications in math- 
ematics and engineering. For example, Problem Q] is related and has applications 
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to the subspace segmentation problem known in computer vision (see e.g., (VMS05 
and the references therein). The subspace segmentation problem is used for face 
recognition, motion tracking in videos, and the Generalized Principle Component 
Analysis GPCA VMS05 . Problem[T]is also related and has application to segmen- 
tation and spectral clustering of Hybrid Linear Models (see e.g., [CL09] and the 
reference therein). Compressed sensing is another related area where finite signals 
in C d are modeled by a union of subspaces M — Ui^jVi, with dimVi < s, where 
s « d |CRT06j . 

1.2. The Minimal Subspace Approximation Property. It has been shown 
that, given a family of closed subspaces C, the existence of a minimizing sequence of 
subspaces V° = {Vf, . . . , V°} that solves Problem Q] is equivalent to the existence 
of a solution to the same problem but for / = 1 [ACM08 : 

Theorem 1.1. Problem]]] has a minimizing set of subspaces for any I > 1 if and 
only if it has a minimizing subspace for 1 = 1. 

This suggests the following definition: 

Definition 1.2. A set of closed subspaces C of a separable Hilbert space H has 
the Minimum Subspace Approximation Property (MSAP) if for every finite subset 
FcW there exists an element V £ C such that minimizes the expression e(F, V) = 
I] /eF eP(/, V) over all V G C. More specifically, we will say that C has MSAP(k) 
for some k G N if the previous property holds for all subsets F of cardinality m < k. 

Using this terminology, Problem [T] has a minimizing sequence of subspaces if and 
only if C satisfies the MSAP. 



Remark 1.3. We will see that, in general, MSAP(k + 1) is strictly stronger than 
MSAP{k). 



There are some cases for which it is known that the MSAP is satisfied. For 
example, U = C d and C = {V C H : dimV < s}, Eckhard- Young theorem |EY36j 
implies that C satisfies MSAP. Another example is when H = L 2 (R d ) and C = 
span{0i, . . . , <j) r } is the set of all shift-invariant spaces of length at most r. For this 
last example, a result in |ACHM07) implies that C satisfies the MSAP. However, a 
general approach for the existence of a minimizer had so far not been considered. 
Thus, the main goal of this paper is to provide necessary and sufficient conditions 
on the class C of closed subspaces in % so that C satisfies the Minimum Subspace 
Approximation Property. 

1.3. Organisation. This paper is organized as follows: Section[2]will be devoted to 
the characterization of MSAP in both finite and infinite dimensional Hilbert spaces. 
We reformulate Problem [T] in Section 12.11 The characterization of MSAP is finite 
dimension is obtained in Section 12.31 The infinite dimensional case is derived in 
Section [mi A generalization of finitely generated shift-invariant spaces is discussed 
in Section |3l and a proof of MSAP in these generalized spaces is derived. This gives 
a new and more conceptual proof of the fact that shift-invariant subspaces of length 
< I, for any I satisfy MSAP which was proved in |ACHM07j . 



Let C be a set of closed subspaces of a separable Hilbert space H. Our main 
concern is to find a topological characterization of the MSAP of C. Recall that C 
has MSAP if for every finite subset F C H, there exists a subspace V° G C that 
minimizes the expression 



2. Characterization of MSAP 




(2.1) 



/eF 



over all V G C. 
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2.1. Reformulation in terms of projectors and states. Let H(H) C B(H) 
denotes the set of orthogonal projectors. The subspaces in C will be identified with 
a subset of projectors. Specifically, a subspace V <E C will be identified with the 
orthogonal projector Q = Qy whose kernel is exactly V (i.e., Q = I — Py where 
Pv is the orthogonal projector on V). In this way, we will identify any set of closed 
subspaces C of rl with a set of projectors {Q G n(H) : ker(Q) S C} C n('H). Using 
this identification, we will denote this set of projectors by C as well, and express e 
in (j2~Tj) as 

e(F,V) = $j ? {Q v ) = Y,(Qvf,f), (2.2) 

where 3>f can now be viewed as a linear functional on the set B(H) of bounded 
linear operators on %. 

The choice of the topology to be considered on the set of bounded linear operators 
B{T-L) is of crucial importance for analyzing the optima of the functional $f- Since 
we are looking for problems of existence of minimizers, compacity will be of great 
help. Hence, the weak operator topology will be the right one considered here. 
Namely, recall that operators of norm < 1 form a compact subset for the weak 
operator topology (this is no longer true with the strong operator or the norm 
topologies when the dimension is infinite). However, in infinite dimensional spaces, 
the set or projectors is not closed for the weak operator topology. This will create 
most of the complication that we will have to face in infinite dimension. In the 
sequel, a "closed" subset of B(TL) will always mean closed for the weak operator 
topology. 

From its expression, $f is a continuous linear functional in the weak operator 
topology of B(H). Since the set of orthogonal projectors n(%) is a bounded set in 
the uniform topology of B(H), it is pre-compact in the weak operator topology of 
B(H). Thus it is evident that if the set C C n('H) is weakly closed in B(H), then 
$f attains its minimum (and maximum) for some V° S C, and we get 

Proposition 2.1. If the set C C n(%) is closed in the weak operator topology of 
B(H), then C satisfies MSAP. 

Note that if we let a := J2feF l/ll 2 ^ 0, tncn a l ®F is a state on B{H), i.e., it 
is a linear functional on B(H) which is non- negative on the subset V + (H) of non- 
negative self-adjoint operators of B{Ti), and which equals 1 on the identity operator. 
We can therefore reformulate MSAP as follows: C has MSAP if every state reaches 
its infimum on C. 

2.2. A sufficient geometric condition for MSAP. Clearly, it is not necessary 
for C to be closed in order for C to have the MSAP even if H has finite dimension. 
For example, the theorem below gives a sufficient condition for MSAP in terms 
of the set C + = C + V + (H) where V + {T-L) is the set of non- negative self adjoint 
operators. It also gives a sufficient condition for MSAP in terms of the convex hull 
co(C) of C, i.e., the smallest convex set containing C: 

Theorem 2.2. 

(C = C) => (C+ = C+) => (co(C+) = co(C+)) =► (C satisfies MSAP), 
and the implications are strict in general. 

Proof. Since C is compact and P + ('H) is closed, C + V + (TL) is closed. Hence C + = 
C + V + (H) = C + V+(n) = C+, and the first implication follows. The second 
implication is obvious. For the last implication, let F be a finite set in Tl. Recall 
that <&f is continuous for the weak topology on B(H). We make the following trivial 
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observation: ifQ € B{T-L) is a convex combination of elements Q , Q 2 , . . ., then there 
exists i such that $>f(Q 1 ) < <S>f(Q)- In particular, we have that 

inf $f = inf , 

co(C) C 

and since C is compact, this inhmum is attained for some Rq S C. By continuity 
of $f, we have $f(^o) = hifQeC $f(Q) = mr Qec ^f(Q)- Using the assumption 
co(C + ) = co(C+), and the easily checked equality co(C + ) = co(C) + P + (H), we get 
that co(C) C co(C) + V + (n). Thus we have that R = Q + N for some Q € co(C) 
and AT € P+(H). It follows that 

$f(Qo) < $f(Qo) + *f(JV) - $ F (i?o) < *f(Qo)- 

Hence $f(Qo) = ^f(Ro) = hrfQeco(c) ${Q)- But since Qo £ co(C), there exists 
P eC such that $ F (P ) = $ F (<9o)- This proves that C satisfies the MSAP. 

To see why the last implication cannot be reversed, take C to be the set of 
all finite dimensional subspaces (except the trivial vector space) of some infinite 
dimensional space H. It has MSAP since for all finite sets F in TL, one can find a 
finite dimensional subspace containing it. On the other hand the convex hull of C + 
does not contains 0, while the weak closure of C does. 

To show that the second implication is not an equivalence, let H = I 2 and consider 
the sequence of vectors: v n — ei + e n which weakly converges to ei, and the sequence 
m„ = e 2 + e„ +1 , which weakly converges to e%. For all n > 3, let P n be the projector 
on the space spanned by v n and w n . One checks that the sequence {P n } converges 
weakly to Q = (Pb 1 +Pb 2 )/2, where E\ — span{ei}, andi^ = span{e2}- Moreover, 
since P n = Q + (Pe„ + f > E n+1 )/2, where E n — span{e n }, and E2 — span{e n +i}, we 
conclude that Q < P n for any n. Now define C = {P n , n>3}U {Pe 15 Pe 2 }- The 
closure of C consists of C U {Q}. By the previous remark, Q does not belong to C + , 
so that C + is not closed. But on the other hand Q e co(C), hence C C co(C) which 
implies that 00(C) = co(C). It follows that co(C + ) = co(C+). 

Finally, to see why the first implication cannot be reverse, let H = M 3 and 
consider the set C = C\ U C2 which is the union of the plane C\ = spanjei, and 
the set of lines C2 = Ut,{span{u} : v = e% + ce2, for some c£l}. Then C (identified 
with a set of projectors as described earlier) is not closed (since Q sp an{e 2 } ^ C) but 
C satisfies the MSAP, since if the infimum is achieved by the missing line span{e2}, 
it is also achieved by the plane C\ . □ 

Remark 2.3. For a perhaps more convincing example of why the last implication 
is not an equivalence, let T-L = i 2 , {ej} its canonical basis, and let C be the set of 
subspaces of co-dimension one except for the space spanjei} -1 . In other words, C 
consists of all projectors of rank one except the projector Qi on spanjei}. Clearly 
for any finite subset F there exist (infinitely many) subspaces of C on which $p 
vanishes, i.e., C has MSAP. However, co(C + ) is not closed since Q\ does not belong 
to it, while being in the closure of C. 

2.3. Characterization of MSAP in finite dimension. The fact that the first 
implication in Theorem 12.21 is not an equivalences was shown using an example in 
a finite dimensional space, while the other two examples showing the the other two 
implications are not equivalence were obtained in an infinite dimensional space. It 
turns out that for finite dimensional spaces the last two implications are in fact 
equivalences, and that for a Hilbert space % of dimension d < 00, MSAP is also 
equivalent to MSAP(d- 1) (see Definition [OJ. We have 

Theorem 2.4. Suppose T-L has dimension d. Then the following are equivalent 
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(i) C satisfies MSAP(d-l); 

(ii) C satisfies MSAP; 

(iii) co(C+) = co{C+); 

(iv) C + is closed. 

Proof. Clearly (ii) implies (i), and (iv) implies (iii). By Theorem 12.21 (iii) implies 
(ii). So we are left to show that (i) implies (iv). Since for finite dimensions, the 
set of projectors of rank k is closed, the zero projector belongs to C if and only if 
it belongs to C. Note that the case where C contains the zero projector is trivial, 
since this implies that C + is all of T J+ ('H). We shall therefore omit this case. 
Now let R G C be a non-zero projector, and let F C H be a basis for keri? - 
Since i?o is non-zero, keri?o has dimension < d — 1 so that #(F) < d — 1. Then 
$f(-Ro) = = infQ e c $f(Q)- Since C satisfies MSAP(d-l) there exists Q C C such 
that &f(Qo) = 0. Hence keri? C kerQ , so that Q < Rq, i.e., i? = Qa + N 
for some N e V+{U). Thus, C C C+ = C + T+(H). We get that C + P + (H) C 
C + V+{H) + P+{H) = C+. Since V + (U) is closed and C is compact, it follows that 
C + V + {H) is closed. Hence C+ is closed. □ 



Remark 2.5. For spaces of dimension d < oo, it is easy to show that MSAP(k-l) is 
strictly weaker than MSAP(k) for 1 < k < d — 1. To see this, let C be the set of 
all subspaces of dimension k except for the subspace generated by k independent 
vectors {vi, . . -,v k }. The C has MSAP(k-l), but it does not satisfy MSAP(k). 

2.4. Characterization of MSAP in infinite dimension. Let A be a locally 
convex topological vector space. The example that we have in mind here is B(TL) 
equipped with the weak-operator topology. 

Definition 2.6. We define a closed half-space to be a set of the form i?<ji,o = {% £ 
X : 4>(x) > a}, for some a £ R and <fi an K-linear functional on X. The boundary 
of such half-space is the (affine) hyperplane of equation <j){x) = a. 

For the reminder of this paper, we will use half-space to mean closed half space. 
Using this terminology, a direct application of Hahn-Banach theorem implies that 
the closed convex hull co(E) of a subset E is the intersection of all half-spaces 
containing E. Let us introduce a slightly different notion, called contact hull. 

Definition 2.7. The contact hull T(E) of a subset E of X is the intersection of 
all half spaces containing E and whose boundary intersects E non-trivially. Such a 
half-space is called contact half-space of E. The set of contact half-spaces containing 
E is denoted by T(E). 

Remark 2.8. Note that if the set of all contact half-spaces of E is empty, then 
T(E) = X. Clearly the contact hull is closed and the closed convex hull is contained 
in the contact hull. However, the converse is not true. 

Now, we are ready to prove a geometric characterization of MSAP. Here the 
locally convex space X is B(TL) equipped with the weak-operator topology. 

Theorem 2.9. Let C be a set of projector in B(H). Then C has MSAP if and only 

T(C+) = T(C+). 

Proof. First note that a R-linear functional <fi € B(T-l)* reaches its minimum a G K 
on some subset E if and only if {x, <fi(x) > a} is a contact half-space of E. 

Let us show that T(C+) = T(C+) implies MSAP. Consider some M-linear func- 
tional <£>f for some finite set F. By definition of the weak operator topology on 
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B(H), $f is continuous for this topology. Hence the inhmum m of <1>f on C + co- 
incides with its infimum on C + = C + . But the infimum m on C + is achieved on C, 
since C is compact. Thus, H$ F m G T(C+). But saying that T(C + ) = T(C + ) exactly 
means that the contact half-spaces to C + and those of C + coincide. This means that 
-ff$ F ,m S T(C + ), and hence $f reaches its minimum on C + , and therefore on C. 

To prove the converse, we will need the following well-known (but crucial) ob- 
servation. In terms of states, it says that states which are continuous for the weak 
operator topology are convex combination of vector states, i.e., a convex combina- 
tions of functionals on B{%) of the form <f> = (•/, /) where / 6 H and ||/|| = 1. 

Claim 2.10. For a real or a complex Hilbert space, the restriction to P + (TL) of an 
element 4> G B(H)* which takes non-negative values on V + (H) can be written as 
4>(A) = $ F (A) for some finite set ¥, and for all A G V + (H). 

Proof. Since is a continuous functional on B(H) endowed with the weak operator 
topology, <p must be of the form 

iei 

for some vectors {vi, Wi : i G /} in H, where / is a finite set. Let M := spanju^, Wi : 
iei} and the operator Am ■= HmAUm where LTm is the orthogonal projector 
on M. Define the operator 4>m{Am) = <j>{A) for all A G B(H). Let {a} be an 
orthonormal basis for M. Then <j> M {B) = j^ij ^ 3 ( Be ^ e i) for a11 B e B(M). This 
can be written as tr(Efi), where S is the matrix with entries Xij and f2 is the matrix 
whose entries are (Bej,a). Since (f> is nonnegative on P + ('H) it follows that for any 
nonnegative definite matrix 51 

tr(Efi) = tr(( E ^ E * )0) = trAS*riS, 

where A is the matrix of eigen values of ( s ~^ s ) and S is the matrix of its eigen 
vectors. Therefore, (f>(A) = $ F (A) for all A G V + (H), where the vectors F = 
{fi, fm} are given by f t = a 1 / 2 J2j Sijej. □ 



Now suppose that C has MSAP. Let H^^ a — {x, <p(x) > a} be a contact half-space 

to C + . Note that this implies that <j> > on V+CH). Therefore, on T+CU), 4> = $ F 
for some F. As C has MSAP by assumption, $f reaches its minimum on C, hence 
on C + . Therefore, the above half-space H^^ a is a contact half space to C + . □ 

Corollary 2.11. If C has the MSAP, then T(C+) = co(C+). 

Proof. Clearly co(C + ) C T(C + ). Moreover, as a general fact, co(C + ) = co(C + ). 
And since C has MSAP, the previous theorem implies that T(C + ) = T(C + ). Hence, 
without loss of generality, we can assume that C is compact. 

Now suppose that x G T(C + ) \ co(C + ). By Hahn-Banach's theorem there exists a 
weak-continuous R- functional 4> G B{Ti)* such that <p(x) < <f)(y) for all y 6 co(C + ). 
Let A := infco( C +) <f>. We have that A = mf m r C +) 4> — m f C o(c+) 4> = m fc+ 4> — m fc <\>- 
Since C is compact we have get that H^^x is a contact half-space to C + . In other 
words, -/?</,, a is a contact half-space containing C + , but not x. This is in contradiction 
with the fact that x G T(C+). □ 
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3. 7r(G)-INVARIANT SUBSPACES AND MSAP 

3.1. Projectors of finite corank. Let H denote a separable Hilbert space, and 
let, as before II = 11(H) be the set of projectors on W, equipped with the topology 
of weak convergence. Let denote the set of positive self-adjoint operators of 
norm < 1. Note that II is closed for the strong topology but not for the weak one, 
unless the dimension is finite, in which case II is compact. Namely, we have 

Proposition 3.1. Let li^ (resp. Tl-k) be the set of projectors of rank k (resp. of 
corank k). IfH is infinite dimensional, then 

(1) The weak closure ofHk is the set of positive self-adjoint operators of norm 
< 1 and of rank at most k. 

(2) The weak closure of Tl-k is the set of positive self-adjoint operators q of 
norm < 1 such that qx = x on a subspace V of codimension at most k 
(i.e., dim V 1 - < k). In other words, q £ LT-fc if and only if q is a positive 
operators of norm \\q\\ < 1 such that q > p for some p £ Tl-k- 

To prove Proposition 13.11 we need the following Lemma: 

Lemma 3.2. The operators of rank < k form a weak closed subset of B(H). 

Proof of Lemma. This follows from the fact that an operator x has a rank less 
than or equal to k if and only if all sub-determinants of size k + 1 of x vanish (for 
instance if x is expressed as a matrix in an orthonormal basis), and the fact that 
the sub-determinant functions of any size n are weakly continuous. □ 

Proof of Proposition \3. l\ Note that the two statements are equivalent. Thus we 
only need to prove (i). 

Note that the set of positive operators, the set of operators with norm < 1, and by 
the lemma, operators of rank < k, are weakly closed sets. So we only have to show 
that every positive self-adjoint operator x of norm < 1 and of rank at most k can be 
weakly approximated by projectors of rank < k. To see that, let us express a; as a 
diagonal matrix in an orthonormal basis (ej)j £ N, such that the diagonal coefficients 
x i.i = ti he in [0, 1], and equal if i > k + 1. For every unit vector v, denote by 
p v be the projector on span{w}. For each i £ N, let s$ = \J 1 — tf- Clearly, for 
every i the sequence of unit vectors v n (i) — \U + Sie ni+n ^+i) converges weakly to 
\ti\ei. Hence the sequence of projectors p v „(i) converges weakly to tfp ei . Moreover 
v n (i) _!_ v n (j) for i ^ j. Therefore the sequence x n = Y^i<i<kPv n (i) ^ s a sequence 
of projectors of rank k that converges weakly to x — ^2 1<i<k tfp ei - D 

Note that (by definition) the space LI^ fc is the set of positive self-adjoint q such that 
there exists a projector p of corank at most k such that q > p. As an immediate 
consequence of (2) in the above proposition, we obtain 

Corollary 3.3. We have that IT^ k = n^ fc , so, in particular, the set C of sub spaces 
of dimension at most k has MSAP. 

3.2. The 7r(G)-invariant setting. Let H denotes a Hilbert space (possibly of 
finite dimension), and let G be a locally compact, er-compact group, and let 7r 
be a continuous unitary representation of G, i.e. a continuous homomorphism 
G -> B(H). 

Definition 3.4. Let W be a closed 7r(G)-invariant subspace of H, i.e., Tr(g)w G W 
for all g £ G and w £ W. We define the 7r(G)-dimension of W to be the minimal 
dimension of a subspace V such that 



W — span{7r(p)u; g £ G,v £ V}. 
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Example 3.5. The prototypical example is TL — L 2 (R d ), G = Z d , and n(k) be- 
ing the translation operator by k. Given fa,...,fa S L 2 (R), the space VF = 
span{0i(- — k), ■ ■ ■ , fa(- — k)} is a 7r(G)-invariant subspace of L 2 . Moreover, by con- 
struction, W = span{7r(fc)/; k 6 Z d , / 6 V}, where V — span{0i, • • • , fa}. Hence 
the 7r(Z d )-dimension of W is < r. Such a space is often called a shift-invariant space 
of length r. 

Recall that an operator A S B(TL) is called 7r(G)-invariant if it commutes with 
the representation n, i.e. if n(g)A(w) — A(Tr(g)(w)) for all g S G and tu S TL. Note 
that if p is a projector, and if the representation is unitary, then p is 7r(G)-invariant 
if and only if its range (resp. its kernel) is a 7r(G)-invariant subspace. 

In the "7r(G)-invariant setting" , we can extend the notion of corank in the obvious 
way, namely by saying that a 7r(G)-invariant operator has 7r(G)-corank k if its 
kernel has 7r(G)-dimension k. We will denote the set of projectors of corank < k by 

n G ,(w). 

Remark 3.6. Note that if the representation ir is trivial (i.e. if n(g) = Id for 
all g G G), then the 7r(G)-dimension and 7r(G)-corank coincide with the usual 
dimension and corank. 

Let us end this section by stating an analogue of Theorem 12.21 in the tt(G)- 
invariant setting. The proof is the same, using the fact that 7r(G)-invariant operators 
form a weak-closed subalgebra of B(H) (i.e. a Von Neumann subalgebra of B(TL)). 
Let us define (V + ) G (TL) to be the set of all 7r(G)-invariant, positive operators in 
B(TL). 

Proposition 3.7. Suppose that C is a set of ir(G)-invariant subspaces of TL. If 
C + (V + ) G (TL) is closed in the weak operator topology, then C satisfies MSAP . 

3.3. MSAP for invariant subspaces. The problem that we address now is whether 
we can generalize Corollary 13.31 to the "7r(G)-invariant" context. Precisely: does 
the set of all closed 7r(G)-invariant subspaces with 7r(G)-dimension at most k has 
MSAP for all k £ N? For simplicity, we will restrict our attention to the case of 
abelian groups. 

As mentioned earlier, an interesting case is the shift-invariant spaces case in which 
G = 7L d acts by translations on TL = L 2 (K. d ). For this case the 7r(G)-invariant sub- 
spaces are usually called shift-invariant subspaces. Moreover, the 7r(G)-dimension 
is called length in this particular example. We now prove that the fact that shift- 
invariant subspaces of length < I, for any / satisfy MSAP [ACHM07 actually holds 
for any unitary representation of a locally compact abelian group G (note that even 
for the group Z, this yields a lot of new examples). 

Theorem 3.8. Let 7r be a unitary representation of a locally compact a-compact 
abelian group G. Then, 

W k cn G k + (v + f(TL). 

In particular, the set of closed tt (G) -invariant subspaces ofTL of ~k(G)- dimension at 
most k has MSAP. 

Our main tool for proving this theorem, will be the generalized spectral theorem 
for locally compact cr-compact abelian groups (see for instance |Dixj ) . Let G be the 
dual of G, i.e. the set of complex characters % on G- Recall that a character is a 
continuous homomorphism from G to the group of complex numbers of modulus 1. 
As this group identifies as £7(1), i.e. the group of unitary operators in dimension 
1, one sees that G can be interpreted as the set of one-dimensional representations 
of G. Note that as U(l) is a compact group, G comes naturally equipped with the 
structure of a locally compact, cr-compact group. To keep in mind some concrete 
examples, let us recall that the dual of Z is U(l), that the dual of (7(1) is Z and 
that R = R. 
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3.4. Proof in finite dimension. Although the proof in finite dimension is not 
different in spirit from the general case, we decided to add it for pedagogical reasons, 
since this allows us to avoid all the problems concerning measurability. 

Let us start by recalling the content of the spectral theorem in finite dimension. 
Assume dim(H) = d. Then there exists an orthonormal basis (ei, . . . , e^) where all 
operators ir(g) are diagonal in this basis. In other words there exists an isometry 
T £ 2 ({l, . . . , d}) such that TTi(g)T- 1 = diag(x 4 (ff)), where for each 1 < i < d, 

Xi is a character of G. Although this description of 7r is quite nice, it could be 
improved to take into account the fact that various Xi s might be equal. We now 
give another description. 

The spectral theorem in finite dimension. Now, for each x G G, let us define 
the multiplicity m x of \ m n to be the number of distinct values of i such that 
X = Xi- Let [i be the measure on G which equals a dirac on x if m x -> 0; an( i 
zero otherwise. For each %, associate the space H x — C m * (note that for all except 
finitely many this space is reduced to zero). Now, define the space L 2 (G, (% X ),/Lt) 
to be the direct sum X % x . An element of this space is denoted v = {v(x)) x snch 
that v(x) 6 7~L X for each x £ G. We equip L 2 (G, (TL X ), /i) with the following scalar 
product: for v, w G L 2 (G, ("H x ), /i), we have 

(v,w) := / (v(x)w{x))n x dn(x) =y2(v(x),w(x))n x - 
Jg x 

Let us now collect a few observations. 
Proposition 3.9. 

(1) There is an isometry S : H — > L 2 (G, /i) such that for all g £ G, and 
allv G L 2 (G,(H x ),n), 

S7r(g)S- 1 (v)(x)=x(9Hx)- 

To simplify the notation, let us identify % with L 2 (G, (H x ),ii) and ir with 
SirS^ 1 , so that we now simply have 

■*(g){v)(x) = x(g)v(x) 

(2) A € 23("H) is 7r(G) -invariant if and only if A decomposes as Q) xe Q A x , with 
A X €B{U X ). 

(3) Suppose that V is a subspace ofH, and let W be the it (G) -invariant subspace 
generated by V, i.e. W — span{i:{g)v; g G G,v 6 V}. Then W = V x 
where each V x is the projection of V onto T-L x . 

Proof. The proofs of (1) and (2) are elementary. Let us only indicate how to prove 
(3). Note that since span{7r(g)v; g G G,v G V"} and that @ x V x is 7r(G)-invariant 
we have that W C X V x . Moreover, since V C W, V x C W, hence X V x C W. 
Thus, W = ®V X . 1 □ 



Let us go back to the purpose of this section, namely the proof of Theorem 13.81 
when H is finite-dimensional. By the above proposition, elements in n_ fc are exactly 
elements of the form p = X P X , where each p x is a projector on TL X of corank at 
most fc, i.e. an element of IT_fc('H x ). Since H is finite, there are only finitely x f° r 
which p x is the nonzero projector. This latest fact together with the fact that for 
each x, the set H-k(H x ) is closed, implies that LI_ fc is closed, and Theorem 13.81 
follows when % is finite-dimensional. 
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3.5. Proof of Theorem 13.81 As previously mentioned, the proof of Theorem 
relies on the generalized spectral theorem for abelian groups that we now recall. 
The first two assertions of the following theorem follow from the classical theory 
(see for instance [Dix]). 

Theorem 3.10 (Generalized spectral theorem for abelian groups). Let G be a 
locally compact a-compact abelian group, and let 7r be a unitary representation of 
G. There exists a Borel measure /i on G, together with an isometry from TL to 
L 2 (G, (H x ),[i), where (%x) x eG * s a measurao ^ e family of subspaces of a separable 
Hilbert space TL' , such that the representation tt decomposes as a direct integral of 
representations ('H x ,7r x ) 7 where tx x is the multiplication by the character \. This is 
usually denoted by 

r® 

tt = 7T x dn(x). 
JG 

Moreover, the following statement are true. 

(i) A G B(TL) is ir(G) -invariant if and only if A decomposes as J g A x d/j,(x) , 
with A x G B{T-L X ) (defined a.e.). 

(ii) Suppose that V is a closed subspace of %, and let W be the closed tt(G)- 
invariant subspace generated by V, i.e. W — span{ir(g)v; g G G,v G V}. 
Then W — J x V x d/i(x) where each V x is the projection of V onto H x 
(defined a.e.). 

(hi) For every x G G, let W x be a subset of operators of B(TL X ) of norm < 1. 
Suppose moreover that the family (W x ) £( j is measurable. Consider W the 

subset of all (n(G) -invariant) operators of the form Jg, A x d/i(x) such that 
A x G W x , which we denote by 



W := / W x d^{x) 



G 



Then, we have 



for the weak topology. 



W= I W x d^( X ), 



G 



Proof of (Hi). Assume B G W, then there exists A n G W such that A n converges 
weakly to B. Since B is also 7r(G)-invariant, B = / £( j B x dfi(x). Thus we have 

/® e<5 ((A£ - B x )v( X )Mx)) n Mx) ^ 0, as n v oo. Hence \\{A^ - B x )v( X )\\n x 
converges in measure to zero, from which we deduce that there exists a subsequence 
(A x — B x )v(x) that converges strongly to a.e.. Hence A x converges weakly to 
B x , a.e.. Thus, B x G W x . Conversely, assume that a sequence A™ G W x converges 
weakly to B x . Then the Lebesgue dominated convergence theorem implies that 
A " = Ixec A xMx), converges weakly to B = /® g6 B x d^{ X ). □ 

The main step toward the proof of Theorem 13.81 is the following description of 
7r(G)-invariant projectors. 

Lemma 3.11. With the notation of Theorem \ 3.8\ we have that for every tt(G)- 
invariant projector p, there exists a measurable field (p x ) of projectors such that 



G 



Moreover, the ir(G)-corank of p is at most k (i.e., kernel of p is n(G)- dimension 
< k) if and only if for a.e. x £ G, corank(p x ) < k. 
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Proof. By (ii) that if p has 7r(G)-corank at most k, then for a.e. x £ G, p x has 
corank at most k. The converse is also true but more subtle. 

Claim 3.12. Let {p x } x ex be a measurable family of projectors of finite corank 
in some separable Hilbert space H' , where X is some Borel measure space. For 
i = 1,2,..., there exists a measurable family of vectors {ef} such that for a.e. 
x G X, (ef , ef , ■ ■ • , &% ) is a basis of kerp^, and ef = for i > r x . 

Proof. First, note that r x — corank(p x ) is a measurable map X — > N, and then 
define a measurable partition of X = Xq U X\ . . . such that r x equals i on Xi for 
each i £ N. So we can suppose that r x is constant, equal to some integer k. 

Now, let us define a metric on X, by pulling back the norm on II = H(H'), i.e. 
d{x, y) := \\p x — Py\\- Clearly, d is measurable. Since II_fe is a separable metric space, 
we can cover it by a countable family of balls of arbitrarily small radius. Hence, one 
sees by an easy argument that it is enough to define {ef } on a small ball, say B(x, e) 
for some x G X, and e > 0. As p x is defined almost everywhere, either B(x,e) has 
measure 0, in which case, there is nothing to do, or there exists y e B(x,e) such 
that p y is well defined, and r y — k. Take an orthonormal basis (ei, e2, . . . , e^) of 
kerp y , and for all z € B(x, e), define ef = (1 — p z )(ei), for i = 1, . . . , k. If e is small 
enough, it is easy to see that the (ef)$ are still linearly independent, and therefore 
form a basis of ker p z . □ 

Note that in the above claim, we can even suppose that the (ef )i<i<r x form an 
orthogonal basis of kerp x for a.e. i£G, and that ||ef || < 1 for a.e. x G kerp x and 
all i. Now, if p = jjf p x dfj,(x), with corank(p x ) < fc, by the claim, we can find a 
measurable family of vectors {(ej , e\ , . . . , e£)} x such that for a.e. \t ( e i > ■ ■ ■ > e fe) 
generates kerp x , and are mutually orthogonal (some might be equal to 0), and that 
||ef || < 1 for a.e. x- Now, let / be a positive function in L 2 (G,/i), and define for 
a.e. x, &(x) = f(x) e f- Note that & G L 2 (G,(H X )) for all i We need to prove 
that kerp is generated, as a closed 7r(G)-invariant subspace by £i, . . . , But since 
p = fg p x dn(x), if W = kerp then VF = J® V^cfy^x), where V x — kerp x . But by 
construction, V x = span{^(x) : i = %,■■■ ,k}. Thus, kerp has 7r(G)-codimension 
k. This finishes the proof of the lemma. □ 

Now we are in position to finish the proof of Theorem 13.81 By Theorem 13.51 (i) , 
every 7r(G)-invariant projector decomposes as p = J^p x dfj,(x) where {p x ) x£ q is a 

Borel measurable family of projectors in L 2 (G, (H x )). The proof of the theorem 
follows immediately from (iii), Corollary 13.31 and the previous lemma. □ 

3.6. Concrete examples and applications. In this section we provide several 
examples and applications to illustrate the results. 

Example 3.13. As a first application of Theorem 12.21 or Corolloarv 13.31 we get the 
well known C. Eckart and G. Young Theorem |EY36I . Specifically, if H is finite 
dimensional and if and C is the family of subspaces of dimension less than or equal 
to r, then C satisfies MSAP since C is closed. As a consequence, Problem [1] has a 
solution. 

Example 3.14. A more interesting example is a generalization of the C. Eckart and 
G. Young Theorem [EY36j when % is infinite dimensional and C is the family of 
subspaces of dimension less than or equal to r. For this case, a direct application 
of Corollarv l3.3l implies that C satisfied MSAP. 
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Example 3.15. Another interesting example is when G = Z, H = -L 2 (1R), 7r(Z)/ := 
Tif where 7] is the translation operator by the translation I, and Ck is the set ol all 
subspaces that are 7r(Z)-invariant and and have 7r(Z)-dimension n. The set Ck is 
precisely the so called shift invariant subspaces of L 2 (R) of length at most fc, that 
acts by translations on H = L 2 (M. d ). As a direct consequence of Theorem 13.81 the 
set Ck satisfies MASP. This gives a new proof for the fact that Ck satisfies MSAP 
IACHM07) . 

Example 3.16. A discrete version of the shift invariant space result above can be 
obtained by letting G — raL where m is a positive integer, % = ^ 2 (Z), Tr(ml)f :— 
S m ix where S m i is the shift operator by the index ml,i.e., if y — S m ix, then y n — 
x-n-rni for all n £ Z. If we let Cfc to be the set of all subspaces that are 7r(mZ)- 
invariant and have 7r(mZ)-dimension k. Then the set Ck satisfies MSAP. 

Example 3.17. An interesting and useful case is in the finite dimensional case % = 
£ 2 ({1, - ■ ■ , a 1 }), where d = ml for some positive integers m and I, Let G = Z/mZ be 
the cyclic group of integers modulo m. By letting Tr(j)x = Sjix be the shift operator 
modulo d we get that the set Ck of all subspaces that are 7r(Z/TOZ)-invariant and 
have 7r(mZ)-dimension k satisfy MASP. 

The last three examples are related in a way similar to the way the Fourier 
Integral, the Fourier series and the discrete FFT are related, and hence this relation 
can be used for numerical approximations and implementations. 
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